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ENERGY MINIMIZATION FOR DATA MERGING AND FUSION 

CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims priority of PCT International application number 
PCT/US98/27374 filed December 23, 1998 and designating the United States, 
which claims priority of U.S. provisional application serial number 60/071,592, 
filed December 29, 1997. 

BACKGROUND OF THE INVENTION 

The present invention relates generally to an iterative method for merging 
or fusing data acquired from varying sources or sensors. More particularly, the 
present invention relates to an iterative method of merging ordinal data which 
results in a ratio measurement level merged value. 

Ordinal data is data which defines an ordering or ranks data in some order. 
In general, some data point is ranked first, another is ranked second, yet another is 
ranked third, and so on. The ranking is done with respect to some established 
criterion or criteria. The ranking may be based on an objective measurement or 
may be made subjectively. 

Ordinal data while relatively uncommon in the physical sciences, is 
ubiquitous in the social sciences, psychology, psychometrics, economics, business 
and marketing and, indeed, in any activity or field of inquiry where data is 
gathered through means other than direct measurement. This includes data 
generated through surveys, expert assessments, preference ratings and rankings 
and through myriad other assessments of ordering, ranking or rating. A very 
simple example is the following: it is common practice in market research to 
assess customer product or service satisfaction through ratings surveys. In these 
surveys, customers are asked to rate some product or service, often on a scale of 1 
to 5, across a number of categories or input domains. The result is a large set of 
ordinal data or alternatively, a set of sets of ordinal data. In general, further 
processing is required to make the ordinal data meaningfully usable. In particular, 
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it is often desirable or required that the ordinal data be merged in some fashion 
and that this merged value be amenable to further quantitative or mathematical 
manipulation. The latter requirement means that the ordinal data must be 
converted to a more measurement-like or quantitative form. More formally, this is 
the problem of generating ratio scale data from ordinal scale data. 

One known analytic technique for merging received ordinal data {x, ,...*„} 
is through computing J a weighted geometric mean 1 " " ^ v " T ^ 

GM(x I ,...xJ = ^^..^;- 

where w = w, + . - . + w „ is the sum of the weights w i associated to the n ordinal 
values x, . The geometric mean is preferred over a weighted mean or average for 
merging ordinal data, since, unlike the average, the geometric mean is invariant 
under order preserving transformations of the data. This means that the order of 
the geometric means, computed over a collection of ordinal data sets, is unaffected 
by order preserving transformations of the raw data. The geometric mean is the 
only meaningful analytic approach to merging ordinal data (Roberts, F. S. in 
Handbooks in OR and MS, vol. 6, eds. S. M. Pollock et al., Elsevier Science, New 
York, 1994). However, the geometric mean does not convert ordinal scale data to 
ratio scale data. The present invention offers an approximate or iterative 
technique for simultaneously merging and converting ordinal data into a ratio 
measurement level scalar. 

PCT international application number PCT/US98/27374, filed December 
23, 1998 and designating the United States, is incorporated herein by reference. 
The incorporated application discloses an energy minimization technique for 
classification, pattern recognition, sensor fusion, data compression, network 
reconstruction and signal processing. The incorporated application shows a data 
analyzer/classifier which comprises using a preprocessing step, an energy 
minimization step and a post processing step to analyze and classify data. In a 
particular embodiment, the energy minimization is performed using individual 
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differences multidimensional scaling. However, the incorporated application does 
not disclose an effective tool for merging or fiision of ordinal data. 

Accordingly, there is a need for an improved method and apparatus for 
processing ordinal data. 

BRIEF SUMMARY OF THE INVENTION 

By way of illustration only,, a sensor fusion, or merging process comprises v . 
a collection of data structures, sources, or sensors, whose values are fused through 
energy minimization to create compressed representations of the input data 
structures, sources, or sensors. This includes a data fusion or merging process for 
ordinal or higher measurement level data that comprises using energy 
minimization with a collection of input source matrices. The data to be fused is 
data measured at ordinal or higher measurement levels including ranking or rating 
information (for definitions of measurement levels or scales, see Stevens, S. S., 
"On the Theory of scales of measurement" Science, 103, pp. 677-680, 1946). 
The data is mapped to a lower triangular or, equivalently, a hollow symmetric 
matrix (see Detailed Description section for definitions) and is processed by an 
energy minimization technique such as individual differences multidimensional 
scaling (IDMDS). Using the IDMDS output, the data are processed for 
presentation of information in a form usable by a human user or in a form suitable 
for further quantitative analysis. A single data value or vector of values may be 
produced to clearly display the quality, validity or other aspect of the input data. 
The components of this output vector are scalars thus facilitating further 
quantitative manipulation and analysis of the input data. The foregoing discussion 
of one embodiment has been presented only by way of introduction. Nothing in 
this section should be taken as a limitation on the following claims, which define 
the scope of the invention. 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a diagram illustrating components of an analyzer according to the 
first embodiment of the invention; and 
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FIG. 2 through FIG. 10 relate to examples illustrating use of an 
embodiment of the invention for data classification, pattern recognition, and signal 
processing. 

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED 
EMBODIMENTS 

The method and apparatus in accordance with the present invention provide 
an analysis tool with many applications. This tool can be used for data 
classification, pattern recognition, signal processing, sensor fusion, data 
compression, network reconstruction, and many other purposes. The invention 
relates to a general method for data analysis based on energy minimization and 
least energy deformations. The invention uses energy minimization principles to 
analyze one to many data sets. As used herein, energy is a convenient descriptor 
for concepts which are handled similarly mathematically. Generally, the physical 
concept of energy is not intended by use of this term but the more general 
mathematical concept. Within multiple data sets, individual data sets are 
characterized by their deformation under least energy merging. This is a 
contextual characterization which allows the invention to exhibit integrated 
unsupervised learning and generalization. A number of methods for producing 
energy minimization and least energy merging and extraction of deformation 
information have been identified; these include, the finite element method (FEM), 
simulated annealing, and individual differences multidimensional scaling 
(IDMDS). The presently preferred embodiment of the invention utilizes individual 
differences multidimensional scaling (IDMDS). 

Multidimensional scaling (MDS) is a class of automated, numerical 
techniques for converting proximity data into geometric data. IDMDS is a 
generalization of MDS, which converts multiple sources of proximity data into a 
common geometric configuration space, called the common space, and an 
associated vector space called the source space. Elements of the source space 
encode deformations of the common space specific to each source of proximity 
data. MDS and IDMDS were developed for psychometric research, but are now 
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standard tools in many statistical software packages. MDS and IDMDS are often 
described as data visualization techniques. This description emphasizes only one 
aspect of these algorithms. 

Broadly, the goal of MDS and IDMDS is to represent proximity data in a 
low dimensional metric space. This has been expressed mathematically by others 
(see, for example, de Leeuw, J. and Heiser, W, "Theory of multidimensional 
scaling/' in P. R. Krishnaiah r and L. N: Kahal, edsi; Handbootof Statistics; V b'ltX 
North-Holland, New York, 1982) as follows. Let 5 be a nonempty finite set, p a 
real valued function on S x S , 

p:SxS->R. 

pis a measure of proximity between objects in S, Then the goal of MDS is to 
construct a mapping / from S into a metric space (X, d), 

f-S->X, 

such that p(ij) = PiJ a d(f(i)j{j)\ that is, such that the proximity of object i to 
object y in S is approximated by the distance in X between /(/) and f(j) . X is 

usually assumed to be n dimensional Euclidean spaced", with n sufficiently 
small. 

IDMDS generalizes MDS by allowing multiple sources. For k = let S k be 

a finite set with proximity measure p k , then IDMDS constructs maps 

f k :S k -+X 

such that Pk {iJ) = p ijk «^(Z(0,ZO*)),for * = l,...,m. 

Intuitively, IDMDS is a method for representing many points of view. The 
different proximities p k can be viewed as giving the proximity perceptions of 
different judges. IDMDS accommodates these different points of view by finding 
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different maps f k for each judge. These individual maps, or their image 
configurations, are deformations of a common configuration space whose 
interpoint distances represent the common or merged point of view. 

MDS and IDMDS can equivalently be described in terms of transformation 
functions. Let P = (p.. ) be the matrix defined by the proximity p on S x S . Then 
MDS defines a transformation function 

where d tj {X) = d(f{i\f(j)) , with / the mapping from S -» X induced by the 
transformation function/ Here, by abuse of notation, X = f(S) , also denotes the 
image of S under / . The transformation function / should be optimal in the sense 
that the distances f(p y ) give the best approximation to the proximities p ir This 
optimization criterion is described in more detail below. IDMDS is similarly re- 
expressed; the single transformation /is replaced by m transformations f k . Note, 
these f k need not be distinct. In the following, the image of S k under f k will be 
written X k . 

MDS and IDMDS can be further broken down into so-called metric and 
nonmetric versions. In metric MDS or IDMDS, the transform at ions/( /J are 
parametric functions of the proximities p^p^ ) . Nonmetric MDS or IDMDS 

generalizes the metric approach by allowing arbitrary admissible transformations/ 
( f k ), where admissible means the association between proximities and 

transformed proximities (also called disparities in this context) is weakly 
monotone: 



Pij <Pki implies f(p {j )<f( Pki ). 
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Beyond the metric-nonmetric distinction, algorithms for MDS and IDMDS 
are distinguished by their optimization criteria and numerical optimization 
routines. One particularly elegant and publicly available IDMDS algorithm is 
PROXSCAL See Commandeur, J. and Heiser, W., "Mathematical derivations in 
the proximity scaling (PROXSCAL) of symmetric data matrices," Tech. report no. 
RR-93-03, Department of Data Theory, Leiden University, Leiden, The 
Netherlands. PROXSCAL is aieast squares, constrained majorizatidn^algbnthm " 
for IDMDS. We now summarize this algorithm, following closely the above 
reference. 

PROXSCAL is a least squares approach to IDMDS which minimizes the 
objective function 

a is called the stress and measures the goodness-of-fit of the configuration 
distances d ij {X k ) to the transformed proximities f h (p iJk ) . This is the most 

general form for the objective function. MDS can be interpreted as an energy 
minimization process and stress can be interpreted as an energy functional. The 
w iJk are proximity weights. For simplicity, it is assumed in what follows that 
w ijk = 1 for all i 9 j\ L 

The PROXSCAL majorization algorithm for MDS with transformations is 
summarized as follows. 

1 . Choose a (possibly random) initial configuration X° . 

2. Find optimal transformations f(p 0 ) for fixed distances d 0 (X°) . 

3. Compute the initial stress 

^AX 0 )^±(f(p iy )-d y (X°))\ 
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4. Compute the Guttman transform X of X° with the transformed 
proximities / (p 0 ) . This is the majorization step. 

5. Replace X° with X and find optimal transformations f(p 0 ) for fixed 
distances rf J> (A r °). 

6. Compute a(f,X°). 

7. Go to step 4 if the difference between the current and previous stress is 
less than s , some previously defined number. Stop otherwise. 

For multiple sources of proximity data, restrictions are imposed on the 
configurations X k associated to each source of proximity data in the form of the 

constraint equation = ZW k . 
This equation defines a common configuration space Z and diagonal weight 
matrices W k . Z represents a merged or common version of the input sources, 
while the W k define the deformation of the common space required to produce the 
individual configurations X k . The vectors defined by diag( W k ), the diagonal 
entries of the weight matrices W k , form the source space ^associated to the 
common space Z 

The PROXSCAL constrained majorization algorithm for IDMDS with 
transformations is summarized as follows. To simplify the discussion, so-called 
unconditional IDMDS is described. This means the m transformation functions 
are the same: f x = / 2 = = f m . 

1 . Choose constrained initial configurations X k . 

2. Find optimal transformations f{p ljk ) for fixed distances d^Xl) . 

3. Compute the initial stress 



*»1 i<j 
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4. Compute unconstrained updates^ of X k using the Guttman transform 
with transformed proximities f(p ijk ) . This is the unconstrained 
majorization step. 

5. Solve the metric projection problem by finding X k minimizing 

m 

W';^X'S=%ti(x k -x k yxx t -xj • - 

subject to the constraints X k = ZW k . This step constrains the updated 
configurations from step 4. 

6. Replace X k with X k and find optimal transformations f(p iJk ) for fixed 
distances d^Xf). 

7. Compute a(f,X?>~»K). ' 

8. Go to step 4 if the difference between the current and previous stress is not 
less than s , some previously defined number. Stop otherwise. 

Here, tr(^) and A 1 denote, respectively, the trace and transpose of matrix 

A. 

It should be pointed out that other IDMDS routines do not contain an 
explicit constraint condition. For example, ALSCAL (see Takane, Y., Young, F, 
and de Leeuw, J., "Nonmetric individual differences multidimensional scaling: an 
alternating least squares method with optimal scaling features," Psychometrika, 
Vol. 42, 1977) minimizes a different energy expression (sstress) over 
transformations, configurations, and weighted Euclidean metrics. ALSCAL also 
produces common and source spaces, but these spaces are computed through 
alternating least squares without explicit use of constraints. Either form of 
IDMDS can be used in the present invention. 

MDS and IDMDS have proven useful for many kinds of analyses. 
However, it is believed that prior utilizations of these techniques have not 
extended the use of these techniques to further possible uses for which MDS and 
IDMDS have particular utility and provide exceptional results. Accordingly, one 
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benefit of the present invention is to incorporate MDS or IDMDS as part of a 
platform in which aspects of these techniques are extended. A further benefit is to 
provide an analysis technique, part of which uses IDMDS, that has utility as an 
analytic engine applicable to problems in classification, pattern recognition, signal 
processing, sensor fusion, and data compression, as well as many other kinds of 
data analytic applications. 

Referring now to* FIG. 1, it illustrates an operatidhar block dia ' 
data analysis/classifier tool 100. The least energy deformation analyzer/classifier 
is a three-step process. Step 1 10 is a front end for data transformation. Step 120 
is a process step implementing energy minimization and deformation 
computations — in the presently preferred embodiment, this process step is 
implemented through the IDMDS algorithm. Step 130 is a back end which 
interprets or decodes the output of the process step 120. These three steps are 
illustrated in FIG. 1. 

It is to be understood that the steps forming the tool 100 may be 
implemented in a computer usable medium or in a computer system as computer 
executable software code. In such an embodiment, step 1 10 may be configured as 
a code, step 120 may be configured as second code and step 120 may be 
configured as third code, with each code comprising a plurality of machine 
readable steps or operations for performing the specified operations. While step 
1 10, step 120 and step 130 have been shown as three separate elements, their 
functionality can be combined and/or distributed. It is to be further understood 
that "medium" is intended to broadly include any suitable medium, including 
analog or digital, hardware or software, now in use or developed in the future. 

Step 1 10 of the tool 100 is the transformation of the data into matrix form. 
The only constraint on this transformation for the illustrated embodiment is that 
the resulting matrices be square. The type of transformation used depends on the 
data to be processed and the goal of the analysis. In particular, it is not required 
that the matrices be proximity matrices in the traditional sense associated with 
IDMDS. For example, time series and other sequential data may be transformed 
into source matrices through straight substitution into entries of symmetric 
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matrices of sufficient dimensionality (this transformation will be discussed in 
more detail in an example below). Time series or other signal processing data 
may also be Fourier or otherwise analyzed and then transformed to matrix form. 

Step 120 of the tool 100 implements energy minimization and extraction of 
deformation information through IDMDS. In the IDMDS embodiment of the tool 
100, the stress function a defines an energy functional over configurations and 
transformations. The configurations are further restricted to those which satisfy J 
the constraint equations X k = ZW k . For each configuration X k . the weight 
vectors diag(0; ) are the contextual signature, with respect to the common space, 
of the £-th input source. Interpretation of a as an energy functional is 
fundamental; it greatly expands the applicability of MDS as an energy 
minimization engine for data classification and analysis. 

Step 130 consists of both visual and analytic methods for decoding and 
interpreting the source space Wfrom step 120. Unlike traditional applications of 
IDMDS, tool 100 often produces high dimensional output. Among other things, 
this makes visual interpretation and decoding of the source space problematic. 
Possible analytic methods for understanding the high dimensional spaces include, 
but are not limited to, linear programming techniques for hyperplane and decision 
surface estimation, cluster analysis techniques, and generalized gravitational 
model computations. A source space dye-dropping or tracer technique has been 
developed for both source space visualization and analytic postprocessing. Step 
130 may also consist in recording stress/energy, or the rate of change of 
stress/energy, over multiple dimensions. The graph of energy (rate or change or 
stress/energy) against dimension can be used to determine network and dynamical 
system dimensionality. The graph of stress/energy against dimensionality is 
traditionally called a scree plot. The use and purpose of the scree plot is greatly 
extended in the present embodiment of the tool 100. 

Let S = {S k } be a collection of data sets or sources S k for k = l 9 ... 9 m . 

Step 1 10 of the tool 100 converts each S k e S to matrix form M(S k ) where 
M{S k ) is a/? dimensional real hollow symmetric matrix. Hollow means the 
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diagonal entries of M{S k ) are zero. As indicated above, M(S k ) need not be 
symmetric or hollow, but for simplicity of exposition these additional restrictions 
are adopted. Note also that the matrix dimensionality p is a function of the data S 
and the goal of the analysis. Since M(S k ) is hollow symmetric, it can be 

interpreted and processed in IDMDS as a proximity (dissimilarity) matrix. Step 
1 10 can be represented by the map 

M:S-+H'(R) 9 * 

S k H>M(S k ) 

where H P (R) is the set of p dimensional hollow real symmetric matrices. The 
precise rule for computing M depends on the type of data in S, and the purpose of 
the analysis. For example, if S contains time series data, then M might entail the 
straightforward entry- wise encoding mentioned above. If S consists of optical 
character recognition data, or some other kind of geometric data, then M(S k ) may 
be a standard distance matrix whose y-th entry is the Euclidean distance between 
"on" pixels i and j\ Mean also be combined with other transformations to form 
the composite, (Mof)(S k ) , where F, for example, is a fast Fourier transform 
(FFT) on signal data S k . To make this more concrete, in the examples below M 
will be explicitly calculated in a number of different ways. It should also be 
pointed out that for certain data collections S it is possible to analyze the conjugate 
or transpose S' of S. For instance, in data mining applications, it is useful to 
transpose records (clients) and fields (client attributes) thus allowing analysis of 
attributes as well as clients. The mapping M is simply applied to the transposed 
data. 

Step 120 of the presently preferred embodiment of the tool 100 is the 
application of IDMDS to the set of input matrices M(S) = {M(S k )} . Each 
M(S k ) e M(S) is an input source for IDMDS. As described above, the IDMDS 

output is a common space Z c R n and a source space W. The dimensionality n of 
these spaces depends on the input data S and the goal of the analysis. For signal 
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data, it is often useful to set n = p - 1 or even n = \S k \ where \S k J denotes the 
cardinality of S k . For data compression, low dimensional output spaces are 

essential. In the case of network reconstruction, system dimensionality is 
discovered by the invention itself. 

IDMDS can be thought of as a constrained energy minimization process. 
As discussed above, the stress a is an energy functional defined over 
transformations and configurations in /? " ; the constraints are defined by the 
constraint equation X k = ZW k . IDMDS attempts to find the lowest stress or 

energy configurations X k which also satisfy the constraint equation. (MDS is the 
special case when each W k = / ,the identity matrix.) Configurations X k most 
similar to the source matrices M(S k ) have the lowest energy. At the same time, 
each X k is required to match the common space Z up to deformation defined by 
the weight matrices W k . The common space serves as a characteristic, or 
reference object. Differences between individual configurations are expressed in 
terms of this characteristic object with these differences encoded in the weight 
matrices W k . The deformation information contained in the weight matrices, or, 
equivalently, in the weight vectors defined by their diagonal entries, becomes the 
signature of the configurations X k and hence the sources S k (through M(S k ) ). 
The source space may be thought of as a signature classification space. 

The weight space signatures are contextual; they are defined with respect to 
the reference object Z. The contextual nature of the source deformation signature 
is fundamental. As the polygon classification example below will show, Z- 
contextuality of the signature allows the tool 1 00 to display integrated 
unsupervised machine learning and generalization. The analyzer/classifier learns 
seamlessly and invisibly. Z-contextuality also allows the tool 100 to operate 
without a priori data models. The analyzer/classifier constructs its own model of 
the data, the common space Z. 

Step 130, the back end of the tool 100, decodes and interprets the source or 
classification space output JFfrom IDMDS. Since this output can be high 
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dimensional, visualization techniques must be supplemented by analytic methods 
of interpretation. A dye-dropping or tracer technique has been developed for both 
visual and analytic postprocessing. This entails differential marking or coloring of 
source space output. The specification of the dye-dropping is contingent upon the 
data and overall analysis goals. For example, dye-dropping may be two-color or 
binary allowing separating hyperplanes to be visually or analytically determined. 
For an analytic approach to separating hyperplanes ' using binary dye-dropping see ~ 
Bosch, R. and Smith, J, "Separating hyperplanes and the authorship of the 
disputed federalist papers," American Mathematical Monthly, Vol. 105, 1998. 
Discrete dye-dropping allows the definition of generalized gravitational clustering 
measures of the form 

g (^ jX ) = >^ 

* ' £exp(/> d(x 9 y)) 



Here, A denotes a subset of W (indicated by dye-dropping), Za( x ) > * s ^ e 
characteristic function on A, d(y) is a distance function, and peR . Such 
measures may be useful for estimating missing values in data bases. Dye- 
dropping can be defined continuously, as well, producing a kind of height function 
on W. This allows the definition of decision surfaces or volumetric discriminators. 
The source space W is also analyzable using standard cluster analytic techniques. 
The precise clustering metric depends on the specifications and conditions of the 
IDMDS analysis in question. 

Finally, as mentioned earlier, the stress/energy and rate of change of 
stress/energy can be used as postprocessing tools. Minima or kinks in a plot of 
energy, or the rate of change of energy, over dimension can be used to determine 
the dimensionality of complex networks and general dynamical systems for which 
only partial output information is available. In fact, this technique allows 
dimensionality to be inferred often from only a single data stream of time series of 
observed data. 
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A number of examples are presented below to illustrate the method and 
apparatus in accordance with the present invention. These examples are 
illustrative only and in no way limit the scope of the method or apparatus. 

Example A: Classification of regular polygons 

The goal of this experiment was to classify a set of regular polygons. The 
collection S = '{S? 3 :~::,S£} with data^sets^, % ->S' A V equilatefdtr iahgles; S 5 ~'S 8 ,' 
squares; S 9 - S n , pentagons; and S n - S l6 ; hexagons. Within each subset of 
distinct polygons, the size of the figures is increasing with the subscript. The 
perimeter of each polygon S k was divided into 60 equal segments with the 

segment endpoints ordered clockwise from a fixed initial endpoint. A turtle 
application was then applied to each polygon to compute the Euclidean distance 
from each segment endpoint to every other segment endpoint (initial endpoint 
included). Let x' St denote the i-tfa endpoint of polygon S k , then the mapping M is 
defined by 

S k »[d sl \d sl \---\d^] 

where the columns 

d * = w< > x k >■ d ( x k . *l ). • • • . d « - 4° » / - 

The individual column vectors d. t have intrinsic interest. When plotted as 

functions of arc length they represent a geometric signal which contains both 
frequency and spatial information. 

The 16, 60 x 60 distance matrices were input into a publicly distributed 
version of PROXSCAL. PROXSCAL was run with the following technical 



WO 00/39705 



PCT/US99/08768 



16 

specifications: sources- 16, objects- 60, dimension- 4, model- weighted, initial 
configuration- Torgerson, conditionally- unconditional, transformations- 
numerical, rate of convergence- 0.0, number of iterations- 500, and minimum 
stress- 0.0. 

FIG. 2 and FIG. 3 show the four dimensional common and source space 
output. The common space configuration appears to be a multifaceted 
representation of the original polygons. 1 ft forms a simple closed path in four ' ' 
dimensions which, when viewed from different angles, or, what is essentially the 
same thing, when deformed by the weight matrices, produces a best, in the sense 
of minimal energy, representation of each of the two dimensional polygonal 
figures. The most successful such representation appears to be that of the triangle 
projected onto the plane determined by dimensions 2 and 4. 

In the source space, the different types of polygons are arranged, and 
hence, classified, along different radii. Magnitudes within each such radial 
classification indicate polygon size or scale with the smaller polygons located 
nearer the origin. 

The contextual nature of the polygon classification is embodied in the 
common space configuration. Intuitively, this configuration looks like a single, 
carefully bent wire loop. When viewed from different angles, as encoded by the 
source space vectors, this loop of wire looks variously like a triangle, a square, a 
pentagon, or a hexagon. 

Example B: Classification of non-regular polygons 
The polygons in Example A were regular. In this example, irregular 
polygons S = {£,,..., £ 6 } are considered, where S l - S 2 are triangles and S 4 - S 6 
rectangles. The perimeter of each figure S k was divided into 30 equal segments 
with the preprocessing transformation M computed as in Example A. This 
produced 6, 30 x 30 source matrices which were input into PROXSCAL with 
technical specifications the same as those above except for the number of sources, 
6, and objects, 30. 
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FIG. 4 and FIG. 5 show the three dimensional common and source space 
outputs. The common space configuration, again, has a "holographic" or faceted 
quality; when illuminated from different angles, it represents each of the polygonal 
figures. As before, this change of viewpoint is encoded in the source space weight 
vectors. While the weight vectors encoding triangles and rectangles are no longer 
radially arranged, they can clearly be separated by a hypeiplane and are thus 
accurately classified by the analysis tool as presently embodied. - 

It is notable that two dimensional IDMDS outputs were not sufficient to 
classify these polygons in the sense that source space separating hyperplanes did 
not exist in two dimensions. 

Example C: Time series data 

This example relates to signal processing and demonstrates the analysis 
tool's invariance with respect to phase and frequency modification of time series 
data. It also demonstrates an entry-wise approach to computing the preprocessing 
transformation M 

The set S = {£,,..., S I2 } consisted of sine, square, and sawtooth waveforms. 
Four versions of each waveform were included, each modified for frequency and 
phase content. Indices 1-4 indicate sine, 5-8 square, and 9-12 sawtooth frequency 
and phase modified waveforms. All signals had unit amplitude and were sampled 
at 32 equal intervals x, for 0 < x £ In . 

Each time series S k was mapped into a symmetric matrix as follows. First, 
an "empty" nine dimensional, lower triangular matrix T k = (f * ) = T(S k ) was 
created. "Empty" meant that T k had no entries below the diagonal and zeros 
everywhere else. Nine dimensions were chosen since nine is the smallest positive 
integer m satisfying the inequality m{m - 1)/2 > 32 and m(m - 1)/2 is the number 
of entries below the diagonal in an m dimensional matrix. The empty entries in 
T k were then filled in, from upper left to lower right, column by column, by 

reading in the time series data from S k . Explicitly: s* { = /*, , the first sample in S k 
was written in the second row, first column of T k ; s k 2 = /* , the second sample in 
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S k was written in the third row, first column of T k , and so on. Since there were 
only 32 signal samples for 36 empty slots in T k , the four remaining entries were 
designated missing by writing -2 in these positions (These entries are then ignored 
when calculating the stress). Finally, a hollow symmetric matrix was defined by 
setting 

M{S k ) = T k +T k r . 

This preprocessing produced 12, 9 x 9 source matrices which were input to 
PROXSCAL with the following technical specifications: sources- 12, objects- 9, 
dimension- 8, model- weighted, initial configuration- Torgerson, conditionality- 
unconditional, transformations- ordinal, approach to ties- secondary, rate of 
convergence- 0.0, number of iterations- 500, and minimum stress- 0.0. Note that 
the data, while metric or numeric, was transformed as if it were ordinal or 
nonmetric. The use of nonmetric IDMDS has been greatly extended in the present 
embodiment of the tool 100. 

FIG. 6 shows the eight dimensional source space output for the time series 
data. The projection in dimensions seven and eight, as detailed in FIG. 7, shows 
the input signals are separated by hyperplanes into sine, square, and sawtooth 
waveform classes independent of the frequency or phase content of the signals. 

Example D: Sequences. Fibonacci, etc. 

The data set S = {S } ,. ..,S 9 } in this example consisted of nine sequences 
with ten elements each; they are shown in Table 1, FIG. 8. Sequences 1-3 are 
constant, arithmetic, and Fibonacci sequences respectively. Sequences 4-6 are 
these same sequences with some error or noise introduced. Sequences 7-9 are the 
same as 1-3, but the negative 1 's indicate that these elements are missing or 
unknown. 

The nine source matrices M(S k ) = (m*. ) were defined by 
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the absolute value of the difference of the z-th and /-th elements in sequence S k . 
The resulting 10 x 10 source matrices where input to PROXSCAL configured as 
follows: sources- 9, objects- 10, dimension- 8, model- weighted, initial 
configuration- simplex, conditionally- unconditional, transformations- numerical, 
rate of convergence- 0.0, number of iterations- 500, and minimum stress- 0.0. 

FIG. 9 shows dimensions 5 and 6 of the eight dimensional source space 
output. The sequences are clustered, hence classified, according to whether they 
are constant, arithmetic, or Fibonacci based. Note that in this projection, the 
constant sequence and the constant sequence with missing element coincide, 
therefore only two versions of the constant sequence are visible. This result 
demonstrates that the tool 100 of the presently preferred embodiment can function 
on noisy or error containing, partially known, sequential data sets. 

Example E: Missing value estimation for bridges 

This example extends the previous result to demonstrate the applicability of 
the analysis tool to missing value estimation on noisy, real-world data. The data 
set consisted of nine categories of bridge data from the National Bridge Inventory 
(NBI) of the Federal Highway Administration. One of these categories, bridge 
material (steel or concrete), was removed from the database. The goal was to 
repopulate this missing category using the technique of the presently preferred 
embodiment to estimate the missing values. 

One hundred bridges were arbitrarily chosen from the NBI. Each bridge 
defined an eight dimensional vector of data with components the NBI categories. 
These vectors were preprocessed as in Example D, creating one hundred 8x8 
source matrices. The matrices were submitted to PROXSCAL with specifications: 
sources- 100, objects- 8, dimension- 7, model- weighted, initial configuration- 
simplex, conditionality- unconditional, transformations- numerical, rate of 
convergence- 0.0, number of iterations- 500, and minimum stress- 0.00001. 
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The seven dimensional source space output was partially labeled by bridge 
material — an application of dye-dropping — and analyzed using the following 
function 



E* 4 (*)•<*(*, jo"' 



where /? is an empirically determined negative number, d(x,y) is Euclidean 

distance on the source space, and z A , is the characteristic function on material set 

A. , / = 1,2 , where /4, is steel, A 2 concrete. (For the bridge data, no two bridges 

had the same source space coordinates, hence g p was well-defined.) A bridge 

was determined to be steel (concrete) if g p (A } ,x) > g p (A 2 ,x) 

(g p (A^x) < g p (A 2 ,x)). The result was indeterminate in case of equality. 

The tool 100 illustrated in FIG. 1 estimated bridge construction material 
with 90 percent accuracy. 

Example F: Network dimensionality for a 4-node network 

This example demonstrates the use of stress/energy minima to determine 
network dimensionality from partial network output data. Dimensionality, in this 
example, means the number of nodes in a network. 

A four-node network was constructed as follows: generator nodes 1 to 3 
were defined by the sine functions, sin(2x), sin( 2x + f ) , and sin( 2x + *f) ; node 4 
was the sum of nodes 1 through 3. The output of node 4 was sampled at 32 equal 
intervals between 0 and In . 

The data from node 4 was preprocessed in the manner of Example D: the 
y-th entry of the source matrix for node 4 was defined to be the absolute value of 
the difference between the i-th and y-th samples of the node 4 time series. A 
second, reference, source matrix was defined using the same preprocessing 
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technique, now applied to thirty two equal interval samples of the function sin(x) 
for 0 < x < In . The resulting 2, 32 x 32 source matrices were input to 
PROXSCAL with technical specification: sources- 2, objects- 32, dimension- 1 to 
6, model- weighted, initial configuration- simplex, conditionality- conditional, 
transformations- numerical, rate of convergence- 0.0, number of iterations- 500, 
and minimum stress- 0.0. The dimension specification had a range of values, 1 to 
6. The dimension resulting in the lowest stress/energy is the dimensionality of the - 
underlying network. 

Table 2, FIG. 10, shows dimension and corresponding stress/energy values 
from the analysis by the tool 100 of the 4-node network. The stress/energy 
minimum is achieved in dimension 4, hence the tool 100 has correctly determined 
network dimensionality. Similar experiments were run with more sophisticated 
dynamical systems and networks. Each of these experiments resulted in the 
successful determination of system degrees of freedom or dimensionality. These 
experiments included the determination of the dimensionality of a linear feedback 
shift register. These devices generate pseudo-random bit streams and are designed 
to conceal their dimensionality. 

The techniques described above may be further applied to fusion or 
merging of data. Data or sensor fusion or merging is the combining of disparate 
data, possibly from a variety of sources or sensors, into a meaningful 
representative form. Preferably, the result is a data point or set of data which can 
readily be displayed or used for comparison by both human and machine users of 
the system. In particular, the techniques above are extended below to the merging 
of ordinal data. Ordinal data is data which defines an ordering or ranks data in 
some order. For data measured below the ratio scale, the merging process 
described here entails the conversion of such data to the ratio measurement level. 

The basic idea behind using an energy minimization approach for ordinal 
data merging is to treat data sets as relational^ linked, deformable configurations. 
Each data set defines a matrix of lengths (a length matrix) encoding relational 
structure both within and between data sets. In the case of ordinal data sets, such 
"lengths" are defined only up to ordering where the order is a (global) relation on 
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the set of all data sets. Using a variational principle, the technique described 
herein monotonically fuses or merges the configurations representing the data sets 
and measures the configuration by configuration deformation produced by this 
.fusion. This measure of deformation is a vector with scalar components defining 
the fused or merged signature for each data set. The dimension of this fusion 
vector depends, in part, on the level of compression or fusion required of the data. 
Taken together; these fusion vectors define- a pyramid^like merging structure for - ^ 
each data set. 

The fusion or merging process is essentially an instantiation of the general 
three step process described above in conjunction with FIG. 1. The data fusion 
method includes a front end or preprocessing step, step 1 10, a processing step, step 
120, and a back end, step 130. The front end entails the creation of the length 
matrices. The process step, in the illustrated embodiment, entails the application 
of individual differences multidimensional scaling (IDMDS) to the length 
matrices. The back end is simply the recording of the merging, fusion vectors 
associated to the data sets. These are the weight vectors which are the contextual 
signatures described above in connection with step 120 of the tool 100 (FIG. 1). 
The merging process may be iterated over dimension resulting in fusion vectors of 
increasing dimensionality. These vectors can be assembled into pyramid-like 
structures with the apexes of these merging pyramids the fused scalar values for 
each data set. Finally, a single scalar value may be generated by computing the 
norm of a merging vector. 

Let S = {S k } be a collection of data sets S k where k = . For 
simplicity, in what follows the S k are assumed to be vectors with the same 
dimension n for all k. The merging process is applicable, however, to very general 
data structures such as matrices. A matrix is a specific type of data structure. 
Further, a lower triangular matrix as described herein and as known in the art is 
another type of data structure. It is to be understood that in alternative 
embodiments, the input data may be arranged as other data structures. Further, the 
illustrated embodiment is directed toward fusion or merging of ordinal data. The 
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ordinal data contain ranking or rating information. More specifically, the ordinal 
data comprise ranking or rating values for a predetermined characteristic among 
the n input domains. It is to be understood that the described data merging process 
will have many applications with other types of data and may be readily adapted to 
such applications. Other types of data include nominal data, ratio data and interval 
data. The techniques described herein in conjunction with processing ordinal data 
may be suitably applied to other types of data such as those enumerated herein. " 

The data merging process applied to S consists of three steps: The first step 
corresponds to the front end, illustrated in FIG. 1 as step 110. The S k are loaded 
into "empty" lower triangular length matrices T k = (/* ) = T(S k ) . Here, "empty" 
means that initially T k has no entries below the diagonal and zeros everywhere 
else. Since the S k have dimension n s that is, n components, the matrices T k have 
dimension the smallest positive integer / satisfying the inequality /(/- 1)/2 > n 
(since /(/ - 1)/2 is the number of entries below the diagonal in an / dimensional 
matrix). The empty entries in T k are filled in, from upper left to lower right, 
column by column, by reading in the components of S k . For example: sf = /£, , the 
first sample in S k is written in the second row, first column of T k ; s k 2 = /*, , the 
second sample in S k is written in the third row, first column of T k , and so on. In 
this way, the components of S k define lengths or distances in the matrix T k . 
(Note, as in Example B above, the T k also define hollow symmetric matrices 
T k +T k .) If the number of components n is less than the number of available 
entries in the / dimensional matrix T k , then S k is augmented with dummy or 
artificial values to fill the remaining matrix entries. The manner of augmentation 
is described in more detail below. It should be pointed out that there are many 
ways to populate the matrix T k with entries from the data sets S k . For example, 
any consistently applied permutation of the S k can produce T h The method 
described here is not special in any way and any other consistent approach can be 
used to create the T k . 
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The matrix T k may be likened to a drive table which shows travel distances 
between a group of cities on a map. The cities are listed across the top and along 
the left edge of the drive table. Each entry in the drive table lists a distance from 
the city corresponding to the current column to the city corresponding to the 
current row. The diagonal of the drive table is empty or has all zero values, 
because those entries correspond to the distance from a city to itself. The upper 
half of the drive fable; aFove^thV^diagonU/is omitted asTSemg redundant witii the 
lower half Thus, each entry in the matrix T k , which is nominally a rating or a 
ranking of some predetermined characteristic among a plurality of input domains, 
is treated by the illustrated process as a length or distance, but where these lengths 
or distances are only determined up to the ordering of the original data. Of course, 
in the present context there are no objects to instantiate the cities in the above 
drive table, that is, there are no known objects whose interpoint proximity give the 
ordinal values in the data sets S k . 

In the second step of the data merging process, illustrated in FIG. 1 as step 
120, the T k are processed using IDMDS in d = dimensions (d = I , produces a 
scalar merging) where r is either automatically determined or is set by the user. 
IDMDS is the variational principle underlying the illustrated embodiment of the 
data merging process. Other suitable energy minimization methods now known or 
developed in the future may be used. If the S k are vectors of ordinal data then 
non-metric IDMDS is used in the processing step. The result of the IDMDS 
processing is at least a source space, as described above. 

For the third step of the data merging process, illustrated in FIG. 1 as step 
130, the source space output of the IDMDS or other energy minimization process 
is used to interpret the data. The IDMDS source space output from step 120 
contains nonnegative merged or fusion vectors w k eR* for Jt = l,.,.,m and 
d = l,...,r (here, R* denotes the positive orthant of d dimensional Euclidean 
space). If a range of merging dimensions has been selected for each data set, then 
these fusion vectors can be assembled into a pyramid structure. That is, if a single 
merging dimension has been selected, the output will have the form of one or 
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more scalar values. If a two dimensional merging has been selected, the output 
will be a 2 component vector which when stacked above the 1 dimensional scalar 
output produces a typographic inverted V . If a three or more dimension merging 
has been selected, the output will have the form of a flat pyramid with the scalar 
one dimensional value at the apex. Different dimensionalities provide different 
ways to view the output of the process. For particular data, a particular back end 
process may prove advantageous. For example, a scalar merging value can be 
produced either by restricting the source space output to be one dimensional or by 
taking the (Euclidean) norm of higher dimensional source space vectors. In the 
presently preferred embodiment of the invention, an advantage to using the 
magnitude or norm of higher dimensional fusion vectors is the increased 
likelihood that the IDMDS process has converged to a global minimum of the 
stress function. In fact, dimension d = / - 1 is optimal in this regard. Selection of a 
particular back end process is well within the skill of those ordinarily skilled in the 
art. 

The above merging process can be generalized and augmented in a number of 
ways. In one alternative embodiment, the merging process can be generalized by 
allowing for weights on the entries of the length matrices T k . For example, by 
transposing the data S y — in other words, by considering the rows of the matrix 
s = (Si I S 2 1 ■ • • | S m ) , with columns S k — and applying the above process (or more 
generally, an energy minimization process) to the row vectors, a weight can be 
associated with each component of the vectors S k . Specifically, applying the 
invention as described above to the transposed data produces a scalar merged 
value c, , / = l,...,n, for each input domain (row) in S. The T k matrix constructed 

from the vector of merged values c = (c l9 ...c n ) defines proximity weights w* =/* 
for the general IDMDS stress function (note that the T k are identical for all k). 
These weights can then be imposed in the processing step 120, resulting in a 
weighted merging on the vectors S k . Of course, weights can also be supplied a 
priori or through some other procedure. The process described earlier is simply 
the special weighted merging with all weights equal to 1. 
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In another alternative embodiment, in the preprocessing step 1 10, the data S k 
may be augmented with artificial values. For instance, the S k may be consistently 
augmented with missing values, repeated constants, or random values. The S k 
may also be augmented through concatenation of copies of the data values 
themselves; a process called replication. Augmentation of the S h allows energy 
minimization merging to be applied to data sets of differing cardinality and with 
missing values. Moreover, by increasing the number of IDMDS objects, S k 
augmentation improves IDMDS avoidance of local minima by allowing for higher 
dimensional classification spaces. 

In the preprocessing step 1 10 as described above is adapted to data which is 
non-metric or ordinal. In cases where this is not a constraint, there are many other 
approaches to the creation of matrix inputs to the energy minimizing process. 
These are detailed in PCT international application number PCT/US98/27374, 
filed December 23, 1998 and designating the United States, which is incorporated 
herein by reference. For both nonmetric and metric data, it is also possible to 
work with asymmetric length matrix encodings. Of course, for asymmetric 
matrices it is no longer possible to think of matrix entries as lengths or distances 
This does not effect the merging process. 

In sum, the data or sensor fusion process described herein may be 
visualized by imagining a deformable crystal having a number of edges greater 
than or equal to the number of input domains in the input data sources or sensors. 
For ordinal data, each input domain corresponds to a predetermined ordinally 
valued characteristic. The lengths of the edges of the crystal correspond to the 
length data produced from the input data. If the data to be fused is ordinal then the 
lengths correspond to those ordered ratings, rankings or other values of the 
predetermined characteristic. An energy minimization process such as IDMDS 
forces the crystal into a common shape, along with other crystals with the same 
(or possibly other) input domains. The lengths of each edge can contract or 
expand as the crystal deforms, where, in the case of ordinal data, this deformation 
is constrained to be isotonic with respect to the original ordering on the lengths. 
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This deformation is a measure which is referred to as the fusion vector. The 
fusion vector may have one or more ratio scale components that characterize the 
data across several predetermined characteristics. The fusion vector is composed 
of ratio scale data and can be quantitatively manipulated. 

From the foregoing, it can be seen that the illustrated embodiment of the 
present invention provides a method and apparatus for fusion or merging of input 
data. Input data are received and formed into orie or more data structures which 
are matrices in the presently preferred embodiment or the invention. The matrices 
are processed using an energy minimization process such as IDMDS in the 
presently preferred embodiment to produce a an output such as a source space and 
a common space. An output or back end process uses analytical or visual methods 
to interpret the source space and the common space. This technique produces an 
output representative of the input ordinal data which is more manageable by the 
user and more readily permits cross comparison for differing data sources. The 
output data are ratio data rather than ordinal data, and are thus defined relative to 
some predetermined unit of measurement. 

While a particular embodiment of the present invention has been shown 
and described, modifications may be made. For example, PROXSCAL is 
illustrated as one acceptable IDMDS routine for performing the energy 
minimization process. However, PROXSCAL may be replaced by other IDMDS 
routines which are commercially available or are proprietary. It is therefore 
intended in the appended claims to cover all such changes and modifications 
which fall within the true spirit and scope of the invention. 
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CLAIMS 



of: 



1 . A data fusion method comprising the steps of: 

(a) receiving data; and 

(b) applying energy minimization to produce fusion vectors. 

2. The data fusion method of claim 1 wherein step (b) comprises steps 

(b 1 ) creating a plurality of data structures using the received data; 

(b2) fusing -the plurality of data structures created from the received data 
using energy minimization. 



3. The data fusion method of claim 2 wherein step (b) comprises 
producing fusion vectors from the plurality of data structures during fusion of the 
plurality of data structures. 

4. The data fusion method of claim 2 wherein step (b) comprises 
compressing the plurality of data structures to produce a new data structure, the 
new data structure having fewer elements than any data structure of the plurality 
of data structures. 

5. The data fusion method of claim 2 wherein the plurality of data 
structures comprise one or more matrices. 

6. The data fusion method of claim 5 wherein the one or more 
matrices comprise lower triangular matrices filled with values of the received data. 

7. The data fusion method of claim 1 further comprising the steps of: 
creating a matrix using received data; 
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transposing the length matrix to produce data structures; 
applying energy minimization to the data structures to produce a plurality 
of merged values; 

constructing a matrix from the merged values to define a proximity weight 
matrix; and 

applying individual differences multidimensional scaling to length matrix 
created with the received data and the proximity weight matrix to 
produce the fusion vectors. 

8. The data fusion method of claim 1 wherein step (b) comprises 
applying individual differences multidimensional scaling to the data. 

9. The data fusion method of claim 1 further comprising the steps of: 
(c) taking the norm of the fusion vectors. 

10. The data fusion method of claim 1 wherein the received data 
comprise ordinal data. 

11. A data fusion method comprising the steps of: 

(a) receiving ordinal data; 

(b) creating matrices of length data using the ordinal data; 

(c) applying energy minimization to the matrices; and 

(e) producing fusion vectors in response to the energy minimization. 



12. The data fusion method of claim 1 1 wherein step (b) comprises the 
steps of: 

(bl) populating lower triangle matrices with elements of a collection of 
sets containing the ordinal data. 



13. 
0*2) 



The data fusion method of claim 1 1 further comprising the step of: 
augmenting the data with artificial values. 
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14. The data fusion method of claim 1 3 wherein step (b2) comprises: 
replacing one or more missing values of the data with artificial values. 

15. The data fusion method of claim 14 wherein step (b2) comprises: 
appending one or more artificial values to the data. 

16. The data fusion method of claim 14 wherein the artificial values 
comprise randomly determined values. 

1 7. The data fusion method of claim 14 wherein the artificial values 
comprise a repeated constant value. 

18. The data fusion method of claim 1 1 wherein the ordinal data 
comprises ranking values for a predetermined characteristic among a plurality of 
input domains. 

19. The data fusion method of claim 1 1 wherein the ordinal data 
comprises rating values for a predetermined characteristic among a plurality of 
input domains. 

20. A data fusion process for data comprising: 

using individual differences multidimensional scaling with one or more 
input length matrices into which the data for fusion has been 
converted to produce at least a source space output; and 

using the source space output to interpret the fused data. 

2 1 . The data fusion process of claim 20 further comprising the steps of: 
prior to the step of individual differences multidimensional scaling, 

receiving ordinal data; and 
producing the one or more input length matrices from the ordinal data. 
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22. The data fusion process of claim 2 1 wherein the producing step 
comprises treating the ordinal data as length data for input to an individual 
differences multidimensional scaling process. 

23 . A data fusion process comprising: 

forming input length matrices' with ordinal 'data; i r v ; * • . - - 

applying an energy minimization process with the input length matrices to 

produce at least a source space output; and 
interpreting the source space output as an indicator of data fusion. 

24. The data fusion process of claim 23 wherein the forming step 
comprises the steps of: 

receiving the ordinal data; 

treating each ordinal datum as a length; 

defining the input length matrices as a square lower triangular matrices of 
sufficient dimension to contain all the ordinal data; and mapping 
consecutive ordinal data into the input length matrices. 

25. The data fusion process of claim 24 wherein the ordinal data 
comprises ranking values for a predetermined characteristic among a 
plurality of input domains. 

26. The data fusion process of claim 24 wherein the ordinal data 
comprises rating values for a predetermined characteristic among a plurality of 
input domains. 

27. The data fusion process of claim 24 wherein the mapping step 
comprises augmenting the data with artificial values. 
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28. The data fusion process of claim 24 wherein the energy 
minimization process comprises non-metric individual differences 
multidimensional scaling using the input length matrices. 

29. The data fusion process of claim 24 further comprising the steps of: 
producing fusion vectors in the source space output; and 

taking a norm ofthe fusion^vectofsv ~* e r: " r 1 " J : ' " J ♦ • — 

30. The data fusion process of claim 24 further comprising the steps of: 
transposing the received ordinal data to produce input length matrices, 

producing a proximity weight matrix; and 

providing the proximity weight matrix to the energy minimization process. 

3 1 . The data fusion process of claim 24 wherein the forming step 
comprises the steps of: 

receiving the data; 
replicating the data; and 

mapping consecutive replicated data into the input length matrices. 

32. Computer executable software program code stored on a computer 
readable medium, the code for data fusion of input data, the code comprising: 

first code that receives data and forms one or more data structures using the 
data; 

second code that applies an energy minimization process to the one or more 

data structures and produces fusion vectors; and 
third code that uses the fusion vectors to provide user output information. 

33. A method for data fusion, the method comprising the steps of: 
receiving one or more sets of ordinal data S k , the ordinal data including 

ranking values for a predetermined characteristic among a plurality 
of input domains; 
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mapping the ordinal data into one or more empty lower triangular length 
matrices T k to define lengths in the one or more matrices T k ; 

processing the matrices T k using an energy minimization process to 
produce output data; and 

processing the output data to provide fused data. 

34. ; The method for data fusion of claim 34 wherein step of processing 
the matrices T k comprises the steps of: 

processing the one or more sets of ordinal data S k in the matrices T k as 

relational^ linked deformable configurations to find a configuration 

corresponding to minimum energy; and 
producing the output data associated with the configuration corresponding 

to minimum energy. 
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